6 research outputs found
Real-World Repetition Estimation by Div, Grad and Curl
We consider the problem of estimating repetition in video, such as performing
push-ups, cutting a melon or playing violin. Existing work shows good results
under the assumption of static and stationary periodicity. As realistic video
is rarely perfectly static and stationary, the often preferred Fourier-based
measurements is inapt. Instead, we adopt the wavelet transform to better handle
non-static and non-stationary video dynamics. From the flow field and its
differentials, we derive three fundamental motion types and three motion
continuities of intrinsic periodicity in 3D. On top of this, the 2D perception
of 3D periodicity considers two extreme viewpoints. What follows are 18
fundamental cases of recurrent perception in 2D. In practice, to deal with the
variety of repetitive appearance, our theory implies measuring time-varying
flow and its differentials (gradient, divergence and curl) over segmented
foreground motion. For experiments, we introduce the new QUVA Repetition
dataset, reflecting reality by including non-static and non-stationary videos.
On the task of counting repetitions in video, we obtain favorable results
compared to a deep learning alternative
Cloth in the Wind: A Case Study of Physical Measurement through Simulation
For many of the physical phenomena around us, we have developed sophisticated
models explaining their behavior. Nevertheless, measuring physical properties
from visual observations is challenging due to the high number of causally
underlying physical parameters -- including material properties and external
forces. In this paper, we propose to measure latent physical properties for
cloth in the wind without ever having seen a real example before. Our solution
is an iterative refinement procedure with simulation at its core. The algorithm
gradually updates the physical model parameters by running a simulation of the
observed phenomenon and comparing the current simulation to a real-world
observation. The correspondence is measured using an embedding function that
maps physically similar examples to nearby points. We consider a case study of
cloth in the wind, with curling flags as our leading example -- a seemingly
simple phenomena but physically highly involved. Based on the physics of cloth
and its visual manifestation, we propose an instantiation of the embedding
function. For this mapping, modeled as a deep network, we introduce a spectral
layer that decomposes a video volume into its temporal spectral power and
corresponding frequencies. Our experiments demonstrate that the proposed method
compares favorably to prior work on the task of measuring cloth material
properties and external wind force from a real-world video.Comment: CVPR 2020. arXiv admin note: substantial text overlap with
arXiv:1910.0786